System And Method For Generating Screen Pointing Information In A Television Control Device

ABSTRACT

A system and method, in a television control device, for generating screen pointing information, substantially as shown in and/or described in connection with at least one of the figures, as set forth more completely in the claims.

CROSS-REFERENCE TO RELATED APPLICATIONS/INCORPORATION BY REFERENCE

This patent application is a continuation application of non-provisionalapplication Ser. No. 12/774,321 filed May 5, 2010 entitled “SYSTEM ANDMETHOD FOR GENERATING SCREEN POINTING INFORMATION IN A TELEVISIONCONTROL DEVICE” which is related to and claims priority from provisionalpatent application Ser. No. 61/242,234 filed Sep. 14, 2009, and titled“TELEVISION SYSTEM,” the contents of which are hereby incorporatedherein by reference in their entirety. This patent application is alsorelated to U.S. patent application Ser. No. 12/774,154, filedconcurrently herewith, titled “SYSTEM AND METHOD FOR GENERATING SCREENPOINTING INFORMATION IN A TELEVISION”; and U.S. patent application Ser.No. 12/774,221, filed concurrently herewith, titled “SYSTEM AND METHODFOR GENERATING TELEVISION SCREEN POINTING INFORMATION USING AN EXTERNALRECEIVER”. The contents of each of the above-mentioned applications arehereby incorporated herein by reference in their entirety.

FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[Not Applicable]

SEQUENCE LISTING

[Not Applicable]

MICROFICHE/COPYRIGHT REFERENCE

[Not Applicable]

BACKGROUND OF THE INVENTION

Present television control devices are incapable of providing pointinginformation to television program viewers. Further limitations anddisadvantages of conventional and traditional approaches will becomeapparent to one of skill in the art, through comparison of such systemswith the present invention as set forth in the remainder of the presentapplication with reference to the drawings.

BRIEF SUMMARY OF THE INVENTION

Various aspects of the present invention provide a system and method, ina television control device, for generating screen pointing information,substantially as shown in and/or described in connection with at leastone of the figures, as set forth more completely in the claims. Theseand other advantages, aspects and novel features of the presentinvention, as well as details of illustrative aspects thereof, will bemore fully understood from the following description and drawings.

BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a diagram illustrating an exemplary television system inaccordance with various aspects of the present invention.

FIG. 2 is a diagram illustrating an exemplary television control devicein accordance with various aspects of the present invention.

FIG. 3 is a diagram illustrating an exemplary television system withon-screen television sensors in accordance with various aspects of thepresent invention.

FIG. 4 is a diagram illustrating an exemplary television system withoff-screen television sensors in accordance with various aspects of thepresent invention.

FIG. 5 is a diagram illustrating an exemplary television system withoff-television sensors in accordance with various aspects of the presentinvention.

FIG. 6 is a diagram illustrating an exemplary television system withtelevision receiver sensors in accordance with various aspects of thepresent invention.

FIG. 7 is a diagram illustrating an exemplary television system withtelevision controller sensors in accordance with various aspects of thepresent invention.

FIG. 8 is a diagram illustrating an exemplary television control devicein accordance with various aspects of the present invention.

FIG. 9 is a flow diagram illustrating the generation of on-screenpointing information in accordance with various aspects of the presentinvention.

FIG. 10 is a flow diagram illustrating the generation of on-screenpointing information in accordance with various aspects of the presentinvention.

DETAILED DESCRIPTION OF VARIOUS ASPECTS OF THE INVENTION

The following discussion will refer to various communication modules,components or circuits. Such modules, components or circuits maygenerally comprise hardware and/or a combination of hardware andsoftware (e.g., including firmware). Such modules may also, for example,comprise a computer readable medium (e.g., a non-transitory medium)comprising instructions (e.g., software instructions) that, whenexecuted by a processor, cause the processor to perform variousfunctional aspects of the present invention. Accordingly, the scope ofvarious aspects of the present invention should not be limited bycharacteristics of particular hardware and/or software implementationsof a module, component or circuit unless explicitly claimed as such. Forexample and without limitation, various aspects of the present inventionmay be implemented by one or more processors (e.g., a microprocessor,digital signal processor, baseband processor, microcontroller, etc.)executing software instructions (e.g., stored in volatile and/ornon-volatile memory). Also for example, various aspects of the presentinvention may be implemented by an application-specific integratedcircuit (“ASIC”) and/or other hardware components.

Additionally, the following discussion will refer to various televisionsystem modules (e.g., television control device modules). It should benoted that the following discussion of such various modules is segmentedinto such modules for the sake of illustrative clarity. However, inactual implementation, the boundaries between various modules may beblurred. For example, any or all of the functional modules discussedherein may share various hardware and/or software components. Forexample, any or all of the functional modules discussed herein may beimplemented wholly or in-part by a shared processor executing softwareinstructions. Additionally, various software sub-modules that may beexecuted by one or more processors may be shared between varioussoftware modules. Accordingly, the scope of various aspects of thepresent invention should not be limited by arbitrary boundaries betweenvarious hardware and/or software components, unless explicitly claimed.

The following discussion may also refer to communication networks andvarious aspects thereof. For the following discussion, a communicationnetwork is generally the communication infrastructure through which acommunication device (e.g., a portable communication device, television,television control device, television provider, television programmingprovider, television receiver, video recording device, etc.) maycommunicate with other systems. For example and without limitation, acommunication network may comprise a cable and/or satellite televisioncommunication network, a cellular communication network, a wirelessmetropolitan area network (WMAN), a wireless local area network (WLAN),a wireless personal area network (WPAN), any home or premisescommunication network, etc. A particular communication network may, forexample, generally have a corresponding communication protocol accordingto which a communication device may communicate with the communicationnetwork. Unless so claimed, the scope of various aspects of the presentinvention should not be limited by characteristics of a particular typeof communication network.

The following discussion will at times refer to an on-screen pointinglocation. Such a pointing location refers to a location on thetelevision screen to which a user (either directly or with a pointingdevice) is pointing. Such a pointing location is to be distinguishedfrom other types of on-screen location identification, such as, forexample, using arrow keys and/or a mouse to move a cursor or to traverseblocks (e.g., on an on-screen program guide) without pointing.

Additionally, the following discussion will at times refer to televisionprogramming. Such television programming generally includes varioustypes of television programming (e.g., television programs, newsprograms, sports programs, music television, movies, television seriesprograms and/or associated advertisements, educational programs, live orrecorded, broadcast/multicast/unicast, etc.). Such televisionprogramming video content is to be distinguished from othernon-programming video content that may be displayed on a televisionscreen (e.g., an electronic program guide, user interface menu, atelevision set-up menu, a typical web page, a document, a graphicalvideo game, etc.). Various aspects of the present invention may, forexample, comprise determining an on-screen pointing location during thepresentation of television programming on the screen of the television.

Turning first to FIG. 1, such figure is a diagram illustrating anon-limiting exemplary television system 100 in accordance with variousaspects of the present invention. The exemplary system 100 includes atelevision provider 110. The television provider 110 may, for example,comprise a television network company, a cable company, amovie-providing company, a news company, an educational institution,etc. The television provider 110 may, for example, be an original sourceof television programming (or related information). Also for example,the television provider 110 may be a communication company that providesprogramming distribution services (e.g., a cable television company, asatellite television company, a telecommunication company, a datanetwork provider, etc.). The television provider 110 may, for example,provide programming and non-programming information and/or videocontent. The television provider 110 may, for example, provideinformation related to a television program (e.g., informationdescribing or otherwise related to selectable objects in programming,etc.).

The exemplary television system 100 may also include a third partyprogram information provider 120. Such a provider may, for example,provide information related to a television program. Such informationmay, for example, comprise information describing selectable objects inprogramming, program guide information, etc.

The exemplary television system 100 may include one or morecommunication networks (e.g., the communication network(s) 130). Theexemplary communication network 130 may comprise characteristics of anyof a variety of types of communication networks over which video contentand/or information related to video content may be communicated. Forexample and without limitation, the communication network 130 maycompare characteristics of a cable television network, a satellitetelevision network, a telecommunication network, the Internet, a localarea network (LAN), a personal area network (PAN), a metropolitan areanetwork (MAN), any of a variety of different types of home networks,etc.

The exemplary television system 100 may include a first television 140.Such a first television 140 may, for example, comprise networkingcapability enabling such television 140 to communicate directly with thecommunication network 130. For example, the first television 140 maycomprise one or more embedded television receivers or transceivers(e.g., a cable television receiver, satellite television transceiver,Internet modem, etc.). Also for example, the first television 140 maycomprise one or more recording devices (e.g., for recording and/orplaying back video content, television programming, etc.).

The exemplary television system 100 may include a first televisioncontroller 160. Such a first television controller 160 may, for example,operate to (e.g., which includes operating when enabled to) controloperation of the first television 140. The first television controller160 may comprise characteristics of any of a variety of televisioncontrolling devices. For example and without limitation, the firsttelevision controller 160 may comprise characteristics of a dedicatedtelevision control device, a universal remote control, a cellulartelephone or personal computing device with television controlcapability, etc.

The first television controller 160 (or television control device) may,for example, transmit signals directly to the first television 140 tocontrol operation of the first television 140. The first televisioncontroller 160 may also, for example, operate to transmit signals (e.g.,via the communication network 130) to the television provider 110 tocontrol video content being provided to the first television 140, or toconduct other transactions (e.g., business transactions, etc.).

As will be discussed in more detail later, the first televisioncontroller 160 may operate to communicate screen pointing informationwith the first television 140 and/or other devices. Also, as will bediscussed in more detail later, various aspects of the present inventioninclude a user pointing to a location on a television screen (e.g.,pointing to an object or person presented in television programming). Insuch a scenario, the user may perform such pointing in any of a varietyof manners. One of such exemplary manners includes pointing with atelevision control device. The first television controller 160 providesa non-limiting example of a device that a user may utilize to point toan on-screen location. The following discussion of FIGS. 2-10 willpresent various non-limiting illustrative aspects of such a televisioncontroller.

The exemplary television system 100 may also include a televisionreceiver 150. The television receiver may, for example, operate toprovide a communication link between a television and/or televisioncontroller and a communication network and/or information provider. Forexample, the television receiver 150 may operate to provide acommunication link between the second television 141 and thecommunication network 130, or between the second television 141 and thetelevision provider 110 (and/or third party program information provider120) via the communication network 130.

The television receiver 150 may comprise characteristics of any of avariety of types of television receivers. For example and withoutlimitation, the television receiver 150 may comprise characteristics ofa cable television receiver, a satellite television receiver, etc. Alsofor example, the television receiver 150 may comprise a datacommunication network modem for data network communications (e.g., withthe Internet, a LAN, PAN, MAN, telecommunication network, etc.). Thetelevision receiver 150 may also, for example, comprise recordingcapability (e.g., programming recording and playback, etc.).

The exemplary television system 100 may include a second televisioncontroller 161. Such a second television controller 161 may, forexample, operate to control operation of the second television 141 andthe television receiver 150. The second television controller 161 maycomprise characteristics of any of a variety of television controllingdevices. For example and without limitation, the second televisioncontroller 161 may comprise characteristics of a dedicated televisioncontrol device, a dedicated television receiver control device, auniversal remote control, a cellular telephone or personal computingdevice with television control capability, etc.

The second television controller 161 may, for example, transmit signalsdirectly to the second television 141 to control operation of the secondtelevision 141. The second television controller 161 may, for example,transmit signals directly to the television receiver 150 to controloperation of the television receiver 150. The second televisioncontroller 161 may additionally, for example, operate to transmitsignals (e.g., via the television receiver 150 and the communicationnetwork 130) to the television provider 110 to control video contentbeing provided to the television receiver 150, or to conduct othertransactions (e.g., business transactions, etc.).

As will be discussed in more detail later, various aspects of thepresent invention include a user pointing to a location on a televisionscreen (e.g., pointing to an object or person presented in televisionprogramming). In such a scenario, the user may perform such pointing inany of a variety of manners. One of such exemplary manners includespointing with a television control device. The second televisioncontroller 161 provides one non-limiting example of a device that a usermay utilize to point to an on-screen location. The following discussionof FIGS. 2-10 will present various non-limiting illustrative aspects ofsuch a television controller.

The exemplary television system 100 was provided to provide anon-limiting illustrative foundation for discussion of various aspectsof the present invention. Thus, the scope of various aspects of thepresent invention should not be limited by any characteristics of theexemplary television system 100 unless explicitly claimed.

Turning next to FIG. 2, such figure is a diagram illustrating anexemplary television control device 200 (e.g., a remote control device)in accordance with various aspects of the present invention. Theexemplary television control device 200 may, for example, share any orall characteristics with the exemplary television control devices 160,161 illustrated in FIG. 1 and discussed previously and/or with any ofthe exemplary television control devices discussed herein.

The exemplary television control device 200 includes a firstcommunication interface module 210. The first communication interfacemodule 210 may, for example, operate to communicate over any of avariety of communication media and utilizing any of a variety ofcommunication protocols. For example, though the first communicationinterface module 210 is illustrated coupled to a wireless RF antenna viaa wireless port 212, the wireless medium is merely illustrative andnon-limiting. The first communication interface module 210 may, forexample, operate to communicate with one or more communication networks(e.g., cable television networks, satellite television networks,telecommunication networks, the Internet, local area networks, personalarea networks, metropolitan area networks, etc.) via which televisionvideo content (e.g., television programming), television controlinformation, and/or other data is communicated. Also for example, thefirst communication module 210 may operate to communicate with localsources of television video content (e.g., video recorders, receivers,gaming devices, etc.). Additionally, for example, the firstcommunication module 210 may operate to communicate with a secondtelevision controller (e.g., directly or via one or more intermediatecommunication networks). Further for example, the first communicationmodule 210 may operate to communicate with a television utilizing any ofa variety of television communication connections and/or protocols(e.g., composite video, component video, HDMI, etc.). Still further, forexample, the first communication module 210 may operate to communicatewith screen pointing sensors.

The exemplary television control device 200 includes a secondcommunication interface module 220. The second communication interfacemodule 220 may, for example, operate to communicate over any of avariety of communication media and utilizing any of a variety ofcommunication protocols. For example, the second communication interfacemodule 220 may communicate via a wireless RF communication port 222 andantenna, or may communicate via a non-tethered optical communicationport 224 (e.g., utilizing laser diodes, photodiodes, etc.). Also forexample, the second communication interface module 220 may communicatevia a tethered optical communication port 226 (e.g., utilizing a fiberoptic cable), or may communicate via a wired communication port 228(e.g., utilizing coaxial cable, twisted pair, HDMI cable, Ethernetcable, any of a variety of wired component and/or composite videoconnections, etc.). The second communication interface module 220 may,for example, operate to communicate with one or more communicationnetworks (e.g., cable television networks, satellite televisionnetworks, telecommunication networks, the Internet, local area networks,personal area networks, metropolitan area networks, etc.) via whichtelevision video content, television control information, and/or otherdata is communicated. Also for example, the second communication module220 may operate to communicate with local sources of television videocontent (e.g., video recorders, other receivers, gaming devices, etc.).Additionally, for example, the second communication module 220 mayoperate to communicate with a second television controller (e.g.,directly or via one or more intervening communication networks). Furtherfor example, the second communication module 220 may operate tocommunicate with a television utilizing any of a variety of televisioncommunication connections and/or protocols (e.g., composite video,component video, HDMI, etc.). Still further, for example, the secondcommunication module 220 may operate to communicate with screen pointingsensors.

The exemplary television control device 200 may also comprise additionalcommunication interface modules, which are not illustrated. Suchadditional communication interface modules may, for example, share anyor all aspects with the first 210 and second 220 communication interfacemodules discussed above.

The exemplary television control device 200 may also comprise acommunication module 230. The communication module 230 may, for example,operate to control and/or coordinate operation of the firstcommunication interface module 210 and the second communicationinterface module 220 (and/or additional communication interface modulesas needed). The communication module 230 may, for example, provide aconvenient communication interface by which other components of thetelevision control device 200 may utilize the first 210 and second 220communication interface modules. Additionally, for example, in anexemplary scenario where a plurality of communication interface modulesare sharing a medium and/or network, the communication module 230 maycoordinate communications to reduce collisions and/or other interferencebetween the communication interface modules 210, 220.

The exemplary television control device 200 may comprise one or moretelevision interface modules 235. The television interface module 235may, for example, operate to manage communications between thetelevision control device 200 and one or more televisions that arecommunicatively coupled thereto (e.g., via the first 210 and/or second220 communication interface modules). For example, the televisioninterface module 235 may operate to communicate general televisionprogramming video information to a television (e.g., while thetelevision control device 200 is operating to determine an on-screenpointing location).

Also, for example, as will be discussed in more detail later, thetelevision interface module 235 may output a signal to the television,television receiver or a second television controller or other devicewith a display, where such signal comprises characteristics adapted tocause the television (or other device) to output a visual indication ofon-screen pointing location. Such an indication may, for example, becommunicated with (e.g., as a part of) other information (e.g., videoinformation, general device control information, etc.) beingcommunicated to the television (or other device), or such an indicationmay be communicated to the television (or other device) independent ofother information.

The exemplary television control device 200 may additionally compriseone or more user interface modules 240. The user interface module 240may generally operate to provide user interface functionality to a userof the television control device 200. For example, and withoutlimitation, the user interface module 240 may operate to provide foruser control of any or all standard television and/or televisionreceiver commands (e.g., channel control, on/off, television outputsettings, input selection, etc.). The user interface module 240 may, forexample, operate and/or respond to user commands utilizing userinterface features disposed on the television receiver (e.g., buttons,touch screen, microphone, etc.) and may also utilize the communicationmodule 230 (and/or first 210 and second 220 communication interfacemodules) to communicate with a television controller, televisionreceiver, another television control device and/or any other televisionsystem component. For example, various user interface features of thetelevision control device 200 may comprise utilization of the television(e.g., utilizing the television screen for menu-driven or other GUIassociated with television, television receiver and/or televisioncontroller operation).

The user interface module 240 may also operate to interface with and/orcontrol operation of any of a variety of sensors that may be utilized toascertain an on-screen pointing location. Non-limiting examples of suchsensors will be provided later (e.g., in the discussion of FIGS. 3-7 andelsewhere herein). For example and without limitation, the userinterface module 240 may operate to receive signals associated withrespective sensors (e.g., raw or processed signals directly from thesensors, through intermediate devices (e.g., a television, televisioncontrol, surround sound system, etc.), via the communication interfacemodules 210, 220, etc.). Also for example, in scenarios in which suchsensors are active sensors (as opposed to purely passive sensors), theuser interface module 240 may operate to control the transmission ofsignals (e.g., RF signals, optical signals, acoustic signals, etc.) fromsuch sensors.

The exemplary television control device 200 may comprise one or moreprocessors 250. The processor 250 may, for example, comprise one or moreof a general purpose processor, digital signal processor,application-specific processor, microcontroller, microprocessor, etc.For example, the processor 250 may operate in accordance with software(or firmware) instructions. As mentioned previously, any or allfunctionality discussed herein may be performed by a processor executinginstructions. For example, though various modules are illustrated asseparate blocks or modules in FIG. 2 for illustrative clarity, suchillustrative modules, or a portion thereof, may be implemented by theprocessor 250.

The exemplary television control device 200 may comprise one or morememories 260. As discussed above, various aspects may be performed byone or more processors executing instructions. Such instructions may,for example, be stored in the one or more memories 260. Such memory 260may, for example, comprise characteristics of any of a variety of typesof memory. For example and without limitation, such memory 260 maycomprise one or more memory chips (e.g., ROM, RAM, EPROM, EEPROM, flashmemory, one-time-programmable OTP memory, etc.), hard drive memory, CDmemory, DVD memory, etc.

The exemplary television control device 200 may also comprise one ormore calibration modules 251 that operate to perform various calibrationactivities. Examples of such calibration activities will be providedlater in this discussion. Briefly, such calibration activities may, forexample, comprise interacting with a user and/or user pointing device(e.g., if different from the television control device 200) to determinesensor signals under known circumstances (e.g., determine sensor signalsin response to known screen pointing circumstances), and processing suchsensor signals to develop algorithms (e.g., transformation matrices,static positional equations, etc.) to determine screen pointing locationbased on sensor signals received during normal operation. As will alsobe discussed later, such calibration may also be utilized to establishsignal gain (or energy) patterns utilized in determining pointinglocation.

The exemplary television control device 200 may comprise one or morelocation-determining modules 252. For example, as will be discussedlater, various on-screen pointing location determinations may compriseprocessing location information. As a non-limiting example, knowing thelocation of a user (e.g., including the location of a pointing device(e.g., which could be the television control device 200) being utilizedby the user) may simplify the solution of various pointing directiondeterminations. For example, knowing exactly where a pointing device islocated (e.g., in three-dimensional space) or where a pointing device islocated along a line (e.g., knowing device location in two-dimensionalspace or land surface coordinates) relative to the television screen(and/or relative to the television control device) may remove a numberof unknown variables from applicable positional equations. Note thatsuch positional information may, in various exemplary scenarios, alsocomprise orientation information for a pointing device (e.g., yaw, pitchand/or roll). Such orientation information may be determined in variousmanners (e.g., through gyroscopic means, sensor alignment with knownreferences, etc.).

The location-determining module 252 may operate to determine user (orpointing device) location in any of a variety of manners. For exampleand without limitation, in an exemplary scenario where the pointingdevice is different from the control device 200, thelocation-determining module 252 may operate to receive locationinformation from the pointing device (e.g., via one of the communicationinterface modules 210, 220). For example, such a pointing device maycomprise positioning system capability (e.g., global positioning system,assisted GPS, cellular or other triangulation systems, etc.) andcommunicate information describing the position of the pointing deviceto the television control device 200. In an exemplary scenario where thetelevision control device 200 is the pointing device, the televisioncontrol device 200 may comprise on-board position-determiningcapability.

Also for example, the location-determining module 252 may (e.g., via theuser interface modules 240) utilize sensor signals to determine theposition (which may include orientation) of the pointing device (or userthereof). For example, signals may arrive at the pointing device atdifferent sensors at different times (or at different phases). Suchtemporal or phase differences may be processed to determine the locationof the pointing device relative to the known location of such sensors.Further for example, the location-determining module 252 may operate tocommunicate pointing device location information with an external systemthat operates to determine the location of the pointing device. Such anexternal system may, for example, comprise a cellular telephonytriangulation system, a home or premises-based triangulation system, aglobal positioning system, an assisted global positioning system, etc.In a non-limiting exemplary scenario where the control device 200 is thepointing device, the location information communicated with the externalsystem may be location information associated with the control device200.

The exemplary television control device 200 may also comprise one ormore sensor processing module(s) 253. As will be explained below, thesensor processing module 253 may operate to receive sensor information(e.g., from the user interface module(s) 240, from the televisioninterface module 235, from the communication interface modules 210, 220,etc.) and process such received sensor information to determine alocation on the television screen to which a user is pointing. Variousexamples of such processing will be provided below. Briefly, suchprocessing may, for example, comprise selecting a sensor with thestrongest signal, interpolating between a plurality of sensors,interpolating between a plurality of sensors having strongest signals,determining gain (or energy) pattern intersections, etc. Various aspectsof the present invention comprise, for example, determining on-screenpointing location during presentation of television programming (e.g.,programming received from a television broadcaster, video recordingdevice, etc.).

Various aspects of the present invention will now be illustrated by wayof non-limiting example. Throughout the following discussion, referencewill continue to be made to the various modules of the televisioncontrol device 200 illustrated in FIG. 2. It should be noted that thefollowing non-limiting examples provide specific examples of variousaspects, and as such, the scope of various aspects of the presentinvention should not be limited by characteristics of any of thespecific examples, unless specifically claimed.

FIG. 3 is a diagram illustrating an exemplary television system 300 withon-screen television sensors in accordance with various aspects of thepresent invention. The television system 300 includes a television 301comprising a television screen 303. The television system 300 alsoincludes a television controller 320 (or other pointing device) pointingto an on-screen pointing location 330 along a line 325 between thetelevision controller 320 and the on-screen pointing location 330. Thetelevision controller 320 may, for example, share any or all aspectswith the exemplary television controllers 160, 161 and 200 discussedpreviously and with all other television controllers discussed herein.The television control device 320 may, for example, be communicativelycoupled directly to the television 301 via a communication link 353. Thetelevision control device 320 may also, for example, be communicativelycoupled directly to the television receiver 350 via communication link352. The television control device 320 may additionally, for example, becommunicatively coupled indirectly to the television 301 via thetelevision receiver 350 through communication links 351 and 352.Accordingly, various aspects of the television control device 320 willbe explained herein with reference to various components of theexemplary television control device 200 illustrated in FIG. 2.

The television system 300 also comprises a television receiver 350 thatis communicatively coupled to the television 301 via a communicationlink 351 (e.g., a two-way communication link providing video informationto the television 301 and/or receiving sensor information from thetelevision 301 for communication to the television control device 320).The exemplary television receiver 350 is also communicatively coupled tothe television controller 320 via a communication link 352.

The exemplary television screen 303 comprises an array of sensorsintegrated into the television screen 303. One of such sensors islabeled sensor 310. Any of a variety of sensor types may be utilized,non-limiting examples of which include light sensors or photo detectors(e.g., photo diodes) and RF sensors (e.g., antenna elements or loops).

The array of sensors may be integrated in the television screen 303 inany of a variety of manners, non-limiting examples of which will now beprovided. For example, the television screen 303 may comprise an arrayof liquid crystal display (LCD) pixels for presenting video media to auser. An array of photo diodes and/or antenna elements may be integratedbetween or behind LCD pixels. For example, every LCD pixel may beassociated with a corresponding photo diode and/or antenna element, orevery N×M block of LCD pixels may be associated with a correspondingphoto diode or antenna element.

As a non-limiting example, an array of photo diodes and/or RF antennaelements may be formed into a substrate beneath or behind transparentLCD substrates. As another example, a photo diode array and/or antennaelement array may be interposed between or behind an array of LCD thinfilm transistors. Also for example, an array of photo diodes and/or RFantenna elements (or other sensors) may be incorporated into atransparent screen overlay. Note that is such an implementation, suchtransparent screen overlay may be installed after-market. For example, auser that has a television control device 320 with the capability todetermine on-screen pointing location may install the transparent screenoverlay. In such an exemplary scenario, there may be one or morecommunication links established between the television control device320 and the sensors in the overlay, where such communication links maybe independent of a communication link over which non-sensor information(e.g., video and/or control information) is communicated between thetelevision 301 and the television control device 320. Such communicationlink may, for example, be adapted to communicate information from eachsensor to the television control device 320 serially (e.g., in atime-multiplexed manner) and/or in parallel.

In a photo detector implementation, passive photo detectors may receivevarying amounts of respective light energy depending on the pointingdirection of a light source (e.g., a light source of the televisioncontrol device 320 or other pointing device) aimed at the screen 303.Also for example, received signals (e.g., pulsed signals) may arrive atdifferent sensors at different respective times/phases (e.g., beingindicative of relative position and/or pointing direction, which mayalso be utilized in a pointing determination). In such a photo detectorimplementation (e.g., utilizing photo diodes), photo detectors may, forexample, be tuned to react to particular light frequencies to reduceinterference from output pixel light and/or associated reflections,ambient light, etc. As a non-limiting example, photo diodes may be tunedto detect light that is not visible to the human eye, visible lightfrequencies that are relatively rare, light patterns that are unlikelyto occur in a television program (e.g., particular pulse codes), etc.

In an antenna element implementation, an array of antenna elements maybe formed on a substrate and placed behind light producing and/orfiltering elements in an LCD screen (e.g., so as to avoid interferingwith emitted light) or may be formed on a transparent substrate withinor in front of the lighted region of the LCD display (e.g., utilizingmicroscopic antenna elements that are too small to significantlyinterfere with light emitted from the display). As discussed above, suchan implementation may be integrated with the television screen 303, butmay also be added as an overlay (e.g., as a production option or anafter-market user or technician installation).

In an RF antenna implementation, passive antennas (or elements of anoverall antenna matrix) may receive varying respective amounts of RFenergy depending on the pointing direction of a directional RF source(e.g., a directional RF source of the television control device 320 orother pointing device) aimed at the screen. Also for example, receivedsignals (e.g., pulsed signals) may arrive at different antennas atdifferent respective times/phases (e.g., being indicative of relativeposition and/or pointing direction, which may also be utilized in apointing determination)

In an exemplary scenario, a user may point a pointing device (e.g., athe television control device 320, a laser pointer, directional RFtransmitter, specifically designed eyewear, a mobile computing device, amobile communication device, a gesture tracking device or glove, etc.)at the television screen 303, where the pointing device directstransmitted energy (e.g., light energy, RF energy, acoustic energy,etc.) at a particular location on the television screen 303 to which thepointing device is being pointed. Note that such transmitted energy willlikely be transmitted directionally and be associated with an intensityor gain pattern with the highest intensity likely at the center of thepattern (i.e., along the pointing line 325) and decreasing as a functionof angle from the center of the pattern (or distance on the screen fromthe on-screen pointing location).

In such an exemplary scenario, each sensor of the array of sensorsintegrated into the screen 303 will likely receive some respectiveamount of energy. For example, the sensor nearest the screen pointinglocation 330 (i.e., along the pointing line 325) will likely receive thehighest amount of energy, sensors adjacent to the screen pointinglocation 330 will likely receive a next highest range of energy, andsensors away from the pointing location 330 will likely receiveprogressively less amounts of energy from the pointing device (e.g., thetelevision control device 320) as a function of distance from thepointing location 330, until such energy is lost in the noise floor.

In such an exemplary scenario, the television control device 320 (e.g.,the user interface module 240 of the television control device 200illustrated in FIG. 2) may receive signals indicative of the energyreceived by the sensors of the sensor array. The television controldevice 320 may receive such signals in various manners, depending on thedegree of integration of such sensors into the television 301. Forexample, in an exemplary scenario where the sensors are fully integratedinto the television screen 303 and operationally integrated into thetelevision 301, the television control device 320 may receive suchsignals via a communication interface between the television controldevice 320 and the television 301 (e.g., via communication link 353, orvia a communication interface between the television 301 and televisioncontrol device 320 via the television receiver 350 (e.g., viacommunication links 351 and 352)). Also for example, in anotherexemplary scenario where the sensors are overlaid on the televisionscreen 303, and where operation of such sensors is independent of thetelevision 301, the television control device 320 may receive suchsignals via a communication link directly between the television controldevice 320 and the sensors, where such a communication link may beindependent of other communication links between the television controldevice 320 and the television 301. Such communication link may, forexample, be adapted to communicate information from each sensor to thetelevision control device 320 serially (e.g., in a time-multiplexedmanner) and/or in parallel.

The user interface module 240 may then, for example, provide informationof such received sensor signals to the sensor processing module 253 forprocessing. The sensor processing module 253 may then, for example,operate to process such information to determine the screen pointinglocation. The sensor processing module 253 may perform such processingin any of a variety of manners, non-limiting examples of which will beprovided below.

For example, the sensor processing module 253 may operate to select thesensor with the highest received energy and determine that the locationof such selected sensor is the on-screen pointing location. For example,in an exemplary scenario where the spatial resolution ofscreen-integrated sensors is relatively fine, such operation mayreliably yield a desired level of accuracy without undue processingoverhead.

In another example, the sensor processing module 253 may operate toselect the sensor with the highest received energy and a plurality ofsensors adjacent to such sensor. Then, for example, the sensorprocessing module 253 may interpolate between the locations of suchsensors (e.g., based, at least in part, on weighting). For example, in afirst dimension in which a sensor to the right of the highest energysensor has a higher received energy than a sensor to the left of thehighest energy sensor, the sensor processing module 253 may determinethat the pointing location is to the right of the highest energy sensor.How much distance to the right may, for example, be determined as afunction of the ratio between respective energies received by the rightand left sensors. Such calculation may, for example, be a linear ornon-linear calculation. Such calculation may also, for example, considerthe expected energy pattern of a transmitting pointing device (e.g., ina scenario where energy fall-off is logarithmic as opposed to linear).

In an additional example, the sensor processing module 253 may operateto select all sensors receiving a threshold amount of energy (e.g., anabsolute threshold level, a threshold level relative to the highestenergy sensor, etc.). Then, for example, the sensor processing module253 may interpolate between the locations of such sensors (e.g., based,at least in part, on respective energy weighting). For example, thesensor processing module 253 may perform non-linear splining betweensensors in a horizontal direction with sensor location on a first axisand sensor energy on a second axis. The sensor processing module 253 maythen operate to select the point on the sensor location axiscorresponding to the peak sensor energy on the vertical axis. Suchsplining and selecting may then be repeated in the vertical direction.Alternatively for example, the sensor processing module 253 may operateto perform multi-dimensional splining to create a surface based onsensor energy and select the highest point on such surface and thecorresponding screen coordinates of such surface.

In a further example, the sensor processing module 253 may operate toselect a first sensor (e.g., the sensor with the highest receivedenergy). Then, for example, the sensor processing module 253 may utilizeinformation of the relative distance between the selected sensor and thepointing device (e.g., the television control device 320), informationof the gain pattern for the signal transmitted from the pointing deviceto the selected sensor, and calibration information to determine wherethe pointing device may be pointed in order for the sensor to receivesuch energy. For example, this may result in a first closed figure(e.g., a circle, cloverleaf, etc.) drawn around the sensor on the screenplane. Then the sensor processing module 253 may repeat the procedurefor a second sensor (e.g., a sensor with the second highest receivedenergy), resulting in a second closed figure. The sensor processingmodule 253 may then, for example, determine the point(s) of intersectionbetween the first and second figures. If only one point of intersectionlies within the border of the screen, then such point of intersectionmight be utilized as an estimate of the pointing location. If, however,there are two potentially significant points of intersection (or moredepending on the figures), then the sensor processing module 253 mayrepeat the procedure for a third sensor (e.g., the sensor with the thirdhighest energy, a sensor generally along the line perpendicular to aline segment between the first and second sensors, etc.) and determine apoint nearest the intersection of the first, second and third closedfigures. Such a point of intersection may then be utilized as anestimate of the pointing location.

The above-mentioned examples of screen-integrated sensors and relatedpointing location determinations were presented as exemplaryillustrations. Though the above-mentioned examples generally discusslight and/or RF energy sensors, other types of sensors may also beintegrated into a television screen or overlaid thereon. For example andwithout limitation, the sensors may comprise acoustic sensors thatoperate to sense acoustic energy (e.g., directed acoustic energydirected to a pointing location on the screen). For example, suchdirected acoustic energy may be formed at frequencies beyond the rangeof human hearing (e.g., and at frequencies beyond the range of pethearing as well).

Also note that various energy radiation patterns may be used, and/or aplurality of energy radiation patterns may be used. For example, though(e.g., for illustrative clarity) the discussion herein generallydiscusses a single energy emission from the pointing device, a pluralityof energy emissions may be utilized. For example and without limitation,a pointing device (e.g., the television control device 320) may transmita plurality of different directed energy emissions (e.g., light, RF,etc.) toward the pointing direction. Also for example, a pointing devicemay transmit one or more energy emissions that move relative to thepointing direction (e.g., in a raster pattern or any other pattern).

After determining on-screen pointing location, the television controldevice 320 may communicate information of such determined location invarious manners. For example and without limitation, the sensorprocessing module 253 of the television control device 200 may utilizethe television interface module 235 to communicate information of suchon-screen pointing location to the television 301 for presentation tothe user. Also for example, the sensor processing module 253 may utilizethe user interface module 240 to communicate information of suchon-screen pointing location to the user (e.g., on a display of thetelevision control device 200). Such communication will also beaddressed in the discussions of FIGS. 9-10.

In addition to various television configurations in which sensors areintegrated into the television screen, sensors may be incorporated intothe television off-screen. Such sensors may, for example, beincorporated in a border around the screen (or overlaid thereon). Forexample and without limitation, FIG. 4 is a diagram illustrating anexemplary television system 400 with off-screen television sensors inaccordance with various aspects of the present invention. The televisionsystem 400 includes a television 401 comprising a television screen 403.The television system 400 also includes a television controller 420 (orother pointing device) pointing to an on-screen pointing location 430along a pointing line 425 between the television controller 420 and theon-screen pointing location 430. The television controller 420 may, forexample, share any or all aspects with the exemplary televisioncontrollers 160, 161, 200 and 320 discussed previously and with allother television controllers discussed herein. The television controldevice 420 may, for example, be communicatively coupled directly to thetelevision 401 via a communication link 453. The television controldevice 420 may also, for example, be communicatively coupled directly tothe television receiver 450 via communication link 452. The televisioncontrol device 420 may additionally, for example, be communicativelycoupled indirectly to the television 401 via the television receiver 450through communication links 451 and 452. Accordingly, various aspects ofthe television control device 420 will be explained herein withreference to various components of the exemplary television controldevice 200 illustrated in FIG. 2.

The television system 400 also comprises a television receiver 450 thatis communicatively coupled to the television 401 via a communicationlink 451 (e.g., a two-way communication link providing video informationto the television 401 and/or receiving sensor information from thetelevision 401 for communication to the television control device 420).The exemplary television receiver 450 is also communicatively coupled tothe television controller 420 via a communication link 452.

The exemplary television 401 comprises an array of sensors integratedinto the television 401 around the border of the screen 403. Three ofsuch sensors are labeled 410, 411 and 412. As discussed above, any of avariety of sensor types may be utilized, non-limiting examples of whichinclude light sensors or photo detectors (e.g., photo diodes), RFsensors (e.g., antenna elements), acoustic sensors (e.g., microphones),etc.

The array of sensors may be integrated around the television screen 403in any of a variety of manners. For example, such sensors may beintegrated in a border of the television screen 403 that is not used foroutputting video content. Such a configuration may, for example, avoidsensor interference with video content being displayed on the screen.Also for example, as illustrated in FIG. 4, such sensors may be mountedto a border material of the television 401.

For example, an array of photo detectors (e.g., photo diodes) and/orantenna elements (e.g., individual antennas or elements of an antennaarray, for example, a phased array) may be incorporated into a border ofthe television 401 around the screen 403. For example, every screenpixel row and/or column may be associated with a pair of correspondingphoto diodes and/or antenna elements, or every N×M block of screenpixels may be associated with one or more corresponding photo diodes orantenna elements (e.g., a row and column sensor, two row and two columnelements, etc.).

In a photo detector implementation, passive photo detectors may receivevarying amounts of respective light energy depending on the pointingdirection of a light source (e.g., a directional light source of thetelevision control device 420) pointed at the screen. Also for example,received signals (e.g., pulsed signals) may arrive at different sensorsat different respective times/phases (e.g., being indicative of relativeposition and/or pointing direction, which may also be utilized in apointing determination). In such a photo detector implementation (e.g.,utilizing photo diodes), photo detectors may, for example, be tuned toreact to particular light frequencies to reduce interference from outputpixel light and/or associated reflections. As a non-limiting example,photo diodes may be tuned to detect light that is not visible to thehuman eye, visible light frequencies that are relatively rare, lightpatterns that are unlikely to occur in a television program (e.g.,particular pulse codes), etc. In one example, the photo detectorsintegrated with the television body off-screen may comprise photo diodesthat operate to detect energy from a laser pointer or directed infraredenergy from a television controller or other pointing device. Note thatanalogously to the on-screen sensors discussed previously, variousaspects may comprise mounting (e.g., adhering) sensors to the televisionbody off-screen. Such sensor installation may, for example, occur at thefactory or after-market by a technician or user.

In an antenna element implementation, an array of antenna elements maybe positioned around the border of the screen 403. In an RF antennaimplementation, passive antennas (or elements of an overall antennamatrix) may receive varying amounts of respective RF energy depending onthe pointing direction of a directional RF source aimed at the screen.Also for example, received signals (e.g., pulsed signals) may arrive atdifferent antennas at different respective times/phases (e.g., beingindicative of relative position and/or pointing direction, which mayalso be utilized in a pointing determination). Note that analogously tothe on-screen sensors discussed previously, various aspects may comprisemounting (e.g., adhering) sensors to the television body off-screen.Such sensor installation may, for example, occur at the factory orafter-market by a technician or user.

In an exemplary scenario, a user may point a pointing device (e.g., aremote controller 420, a laser pointer, directional RF transmitter,specifically designed eyewear, a mobile computing device, a mobilecommunication device, a gesture tracking device or glove, etc.) at thetelevision screen 403, where the pointing device directs transmittedenergy (e.g., light and/or RF energy and/or acoustic energy) at aparticular location on the television screen 403 to which the device isbeing pointed. Note that such transmitted energy will likely betransmitted directionally and be associated with an intensity or gainpattern with the highest intensity likely at the center of the pattern(i.e., along the pointing line 425) and decreasing as a function ofangle from the center of the pattern. Such a gain pattern is generallyrepresented in FIG. 4 by the concentric circles around the on-screenpointing location 430. Note, however, that in practice such a gainpattern is likely to be more complex than the illustrated pattern (e.g.,including lobes with respective peaks and nulls).

In such an exemplary scenario, each sensor of the sensors integratedinto the television 401 around the border of the screen 403 will likelyreceive some respective amount of energy. For example, along aparticular axis, the sensor nearest the screen pointing location 430(i.e., along the pointing line 425) will likely receive the highestamount of energy, sensors along the particular axis adjacent to thescreen pointing location 430 will likely receive a next highest range ofenergy, and sensors away from the pointing location 430 will likelyreceive progressively less amounts of energy from the pointing device(e.g., the television control device 420), as a function of distancefrom the pointing location 430 or as a function of the angulardisplacement from the pointing line 425, until such energy is lost inthe noise floor.

For example, along the horizontal axis, sensor 410 is closest to thepointing location 430 and will likely receive the highest energy, withsensors adjacent to the left and right of sensor 410 receiving the nexthighest amounts of energy, and so on. Also, along the vertical axis,sensors 411 and 412 will likely receive close to the highest amount ofenergy, with sensors above and below such sensors 411, 412 receiving thenext highest amounts of energy and so on.

In such an exemplary scenario, the television control device 420 (e.g.,the user interface module 240 of the television control device 200illustrated in FIG. 2) may receive signals indicative of the energyreceived by the sensors of the television 401. The television controldevice 420 may receive such signals in various manners, depending on thedegree of integration of such sensors into the television 401. Forexample, in an exemplary scenario where the sensors are fully integratedinto the television 401 (e.g., into a border around the screen 403) andoperationally integrated into the television 401, the television controldevice 420 may receive such signals via a communication interfacebetween the television control device 420 and the television 401 (e.g.,via communication link 453 or via a communication interface between thetelevision 401 and the television control device 420 via the televisionreceiver 450 (e.g., via communication links 451 and 452)). Also forexample, in another exemplary scenario where the sensors are overlaid on(e.g., adhered to) the television screen 401, and where operation ofsuch sensors is independent of the television 401, the televisioncontrol device 420 may receive such signals via a communication linkdirectly between the television control device 420 and the sensors,where such a communication link may be independent of othercommunication links between the television control device 420 and thetelevision 401. Such communication link(s) may, for example, be adaptedto communicate information from each sensor to the television controldevice 420 serially (e.g., in a time-multiplexed manner) and/or inparallel.

The user interface module 240 may then, for example, provide informationof such received sensor signals to the sensor processing module 253 forprocessing. The sensor processing module 253 may then, for example,operate to process such information to determine the screen pointinglocation. The sensor processing module 253 may perform such processingin any of a variety of manners, non-limiting examples of which will beprovided below.

For example, the sensor processing module 253 may operate to select thesensor with the highest received energy along each of the horizontal andvertical axes and determine that the respective locations of suchselected sensors correspond to the horizontal and vertical coordinatesof the on-screen pointing location. For example, in an exemplaryscenario where the spatial resolution of screen border sensors isrelatively fine, such operation may reliably yield a desired level ofaccuracy without undue processing overhead. For example, the sensorprocessing module 253 may determine that sensors 410 and 411 have thehighest received energy for the horizontal and vertical axes,respectively, and thus determine that the on-screen pointing location isrepresented in the horizontal axis by the horizontal location of thesensor 410 and represented in the vertical axis by the vertical locationof the sensor 411. Note that in scenarios where two sensors haverelatively similar energy levels (e.g., as might occur at sensors 411and 412), the sensor processing module 253 may select a midpoint betweensuch sensors (e.g., the vertical midpoint between sensors 411 and 412).

In another example, the sensor processing module 253 may operate toselect, for each screen axis, the sensor with the highest receivedenergy and a plurality of sensors adjacent to such sensor. Then, forexample, the sensor processing module 253 may interpolate between thelocations of such sensors (e.g., based, at least in part, on weighting).For example, in the horizontal dimension in which a sensor to the rightof the highest energy sensor 410 has a higher received energy than asensor to the left of the highest energy sensor 410, the sensorprocessing module 253 may determine that the pointing location along thehorizontal axis is to the right of the highest energy sensor 410. Howmuch distance to the right may, for example, be determined as a functionof the ratio between respective energies received by the right and leftsensors. Such calculation may, for example, be a linear or non-linearcalculation. Such calculation may also, for example, consider theexpected energy pattern of a transmitting pointing device (e.g., in ascenario where energy fall-off is logarithmic as opposed to linear). Thesensor processing module 253 may then, for example, repeat suchoperation in the vertical direction.

In another example, the sensor processing module 253 may operate toselect all sensors in each of the axes receiving a threshold amount ofenergy (e.g., an absolute threshold level, a threshold level relative tothe highest energy sensor, etc.). Then, for example, the sensorprocessing module 253 may interpolate between the locations of suchsensors (e.g., based, at least in part, on respective energy weighting).For example, the sensor processing module 253 may perform non-linearsplining between sensors in a horizontal direction with sensor locationon a first axis and sensor energy on a second axis. The sensorprocessing module 253 may then operate to select the point on the sensorlocation axis corresponding to the peak sensor energy on the verticalaxis. Such splining and selecting may then be repeated in the verticalscreen direction. Alternatively for example, the sensor processingmodule 253 may operate to perform multi-dimensional splining to create asurface based on sensor energy and select the highest point on suchsurface and the corresponding screen coordinates of such surface.

After determining on-screen pointing location, the television controldevice 420 may communicate information of such determined location invarious manners. For example and without limitation, the sensorprocessing module 253 of the television control device 200 may utilizethe television interface module 235 to communicate information of suchon-screen pointing location to the television 401 for presentation tothe user. Also for example, the sensor processing module 253 may utilizethe user interface module 240 to communicate information of suchon-screen pointing location to the user (e.g., on a display of thetelevision control device 200). Such communication will also beaddressed in the discussions of FIGS. 9-10.

In addition to various television configurations in which sensors areintegrated into the television off-screen or off the video presentationportion of the screen, sensors may be incorporated into the televisionsystem off-television. Such sensors may, for example, be incorporated inother components of a television system besides the television. Forexample and without limitation, FIG. 5 is a diagram illustrating anexemplary television system 500 with off-television sensors inaccordance with various aspects of the present invention. The televisionsystem 500 includes a television 501 comprising a television screen 503.The television system 500 also includes a television controller 520 (orother pointing device) pointing to an on-screen pointing location 530along a pointing line 525 between the television controller 520 and theon-screen pointing location 530. The television controller 520 may, forexample, share any or all aspects with the exemplary televisioncontrollers 160, 161, 200, 320 and 420 discussed previously and with allother television controllers discussed herein. Accordingly, variousaspects of the television control device 520 will be explained hereinwith reference to various components of the exemplary television controldevice 200 illustrated in FIG. 2. The television control device 520 may,for example, be communicatively coupled directly to the television 501via a communication link (not illustrated). The television controldevice 520 may also, for example, be communicatively coupled directly tothe television receiver 550 via communication link 562. The televisioncontrol device 520 may additionally, for example, be communicativelycoupled indirectly to the television 501 via the television receiver 550through communication links 561 and 562. Accordingly, various aspects ofthe television control device 520 will be explained herein withreference to various components of the exemplary television controldevice 200 illustrated in FIG. 2.

The television system 500 also comprises a television receiver 550 thatis communicatively coupled to the television 501 via a communicationlink 561 (e.g., a two-way communication link providing video informationto the television 501 and/or receiving sensor information from thetelevision 501 for communication to the television control device 520).

The television control device 520 is illustrated with one or morecommunication links 563 to the various sensors 551-556 independent ofother communication links (e.g., links to the television 501, links tothe television receiver 550, etc.). Note that in various exemplaryscenarios, the television control device 520 (e.g., a user interfacemodule 240) may receive sensor information from the television 501 via atelevision communication link (not illustrated), via a communicationlink 562 with the television receiver 550 and/or via the independentcommunication link(s) 563. The exemplary television control device 520may also be communicatively coupled to other pointing devices and/ortelevision control devices.

The exemplary television system 500 comprises an array of sensorsintegrated into audio speaker components (e.g., surround sound speakers)positioned around the television 501. For example, the television system500 comprises a left speaker 531 comprising a top sensor 552 and abottom sensor 551. Also for example, the television system 500 comprisesa right speaker 533 comprising a top sensor 556 and a bottom sensor 555.Additionally for example, the television system 500 comprises a centerspeaker 532 comprising a left sensor 553 and a right sensor 554. Asdiscussed above, any of a variety of sensor types may be utilized,non-limiting examples of which include light sensors or photo detectors(e.g., photo diodes), RF sensors (e.g., antenna elements), acousticsensors (e.g., microphones), etc. Note that the audio speaker componentexample discussed herein is merely illustrative and that such sensorsmay be installed in any of a variety of locations (e.g., dedicatedsensor boxes, attached to furniture, etc.).

The array of sensors may be positioned around the television 501 in anyof a variety of manners. For example, such sensors may be positionedaround the television 501 generally in the same plane as the televisionscreen 503. In such an exemplary scenario, on-screen pointing locationmay be determined in a manner similar to the interpolation and/or gainpattern intersection discussed above with regard to off-screen and/oron-screen sensors. Note that since the locations of the sensors arelikely to be inconsistent between various television systemconfigurations, a calibration procedure may be implemented (e.g., by thecalibration module 251). Such calibration will be discussed in moredetail below.

In an exemplary configuration, one or more photo detectors (e.g., photodiodes) and/or antenna elements (e.g., individual antennas or elementsof an antenna array) may be incorporated into a plurality of respectivesurround sound speakers positioned around the television 501.

For example, in a photo detector implementation, passive photo detectorsmay receive varying amounts of respective light energy depending on thepointing direction of a light source (e.g., a directional light sourceof the television control device 520) aimed at the screen. As discussedpreviously, directed energy (e.g., light, RF, acoustic, etc.) may betransmitted in a pattern (or envelope), so even if a pointing device(e.g., the television control device 520) is pointed to a location onthe television screen 530 along pointing line 525, sensors off-screen(or even off-television) may still receive energy from the transmission(albeit likely not with the same intensity at which energy is deliveredalong the pointing line 525). Also for example, received signals (e.g.,pulsed signals) may arrive at different sensors at different respectivetimes/phases (e.g., being indicative of relative position and/orpointing direction, which may also be utilized in a pointingdetermination).

In a photo detector implementation (e.g., utilizing photo diodes), photodiodes may, for example, be tuned to react to particular lightfrequencies to reduce interference from output pixel light and/orassociated reflections, ambient light, room lighting, etc. As anon-limiting example, photo diodes may be tuned to detect light that isnot visible to the human eye, visible light frequencies that arerelatively rare, light patterns that are unlikely to occur in atelevision program (e.g., particular pulse codes), etc. In one example,the photo detectors integrated with off-television components maycomprise photo diodes that operate to detect energy from a laser pointeror directed infrared energy from a television controller (or otherpointing device). Note that analogously to the on-screen sensorsdiscussed previously, various aspects may comprise mounting (e.g.,adhering) sensors to various off-television components. Such sensorinstallation may, for example, occur at the factory or after-market by atechnician or user.

In an antenna element implementation, an array of antenna elements maybe positioned around off-television components (e.g., in surround soundcomponents). In an RF antenna implementation, passive antennas (orelements of an overall antenna matrix) may receive varying amounts ofrespective RF energy depending on the pointing direction of adirectional RF source (e.g., a directional RF source of the televisioncontroller 520) pointed at a location on the screen. Also for example,received signals (e.g., pulsed signals) may arrive at different antennasat different respective times/phases (e.g., being indicative of relativeposition and/or pointing direction, which may also be utilized in apointing determination). Note that analogously to the on-screen sensorsdiscussed previously, various aspects may comprise mounting (e.g.,adhering) sensors to the off-television components. Such sensorinstallation may, for example, occur at the factory or after-market by atechnician or user.

In an exemplary scenario, a user may point a pointing device (e.g., atelevision controller 520 (e.g., a remote control device), a laserpointer, directional RF transmitter, specifically designed eyewear, amobile computing device, a mobile communication device, a gesturetracking device or glove, etc.) at the television screen 503, where thepointing device directs transmitted energy (e.g., light and/or RF energyand/or acoustic energy) at a particular location on the televisionscreen 503 to which the user is pointing with the pointing device. Notethat such transmitted energy will likely be transmitted directionallyand be associated with an intensity or gain pattern with the highestintensity at the center of the pattern (i.e., along the pointing line525) and decreasing as a function of angle from the center of thepattern (or distance from the center point). Such a gain pattern wasdiscussed previously in the discussion of FIG. 4.

In such an exemplary scenario, each sensor of the sensors integratedinto the television system 500 off-television will likely receive somerespective amount of energy. For example, along a particular axis, thesensor nearest to the screen pointing location 530 (i.e., along thepointing line 525) will likely receive the highest amount of energy, asensor next nearest to the screen pointing location 530 will likelyreceive a next highest range of energy, and sensors away from thepointing location 530 will likely receive progressively less amounts ofenergy from the pointing device (e.g., the television control device420), as a function of distance from the pointing location 530 and/orangle off the pointing line 525 (e.g., until such energy is lost in thenoise floor). For example, sensor 553 is nearest to the pointinglocation 530 and will likely receive the highest energy, sensor 552 isnext nearest to the pointing location 530 and will likely receive thenext highest energy, and so on.

Note that in the implementation illustrated in FIG. 5, in particularsince there are a relatively low number of sensors, signals from a samesensor may be utilized in determining multiple axes of pointinglocation. As mentioned previously, a calibration procedure may beperformed when the system 500 is configured to assist in such pointingdetermination.

In an exemplary scenario, the television control device 520 (e.g., theuser interface module 240 of the television control device 200illustrated in FIG. 2) may receive signals indicative of the energyreceived by the sensors of the television system 500. The televisioncontrol device 520 may receive such signals in various manners,depending on the degree of integration of such sensors into thetelevision 501. For example, in an exemplary scenario where the sensorsare fully integrated into the television system 500 components (e.g.,surround sound speaker components 531-533) and operationally integratedinto such components, the television control device 520 may receive suchsignals via a communication interface between the television controldevice 520 and the respective off-television components (e.g., via acommunication link 563 between the television control device 520 and thesurround sound speaker components 531-533). Also for example, in anotherexemplary scenario where the sensors are overlaid on (e.g., adhered to)the off-television components, and where operation of such sensors isindependent of the television 501, the television control device 520 mayreceive such signals via a communication link directly between thetelevision control device 520 and the individual sensors (e.g.,communication link 563), where such a communication link may beindependent of other communication links between the television controldevice 520 and the television 501 and/or independent of othercommunication links between the television control device 520 and othertelevision system 500 components (e.g., television receiver 550 and thesurround sound speaker components 531-533).

The user interface module 240 may then, for example, provide informationof such received sensor signals to the sensor processing module 253 forprocessing. The sensor processing module 253 may then, for example,operate to process such information to determine the screen pointinglocation. The sensor processing module 253 may perform such processingin any of a variety of manners, non-limiting examples of which will beprovided below.

In an exemplary scenario, the sensor processing module 253 may operateto estimate a position between sensor positions based on relative sensorenergy. For example, in the horizontal dimension, sensor 552 maycorrespond to a relatively high amount of energy, and sensor 556 maycorrespond to a relatively low amount of received energy. The sensorprocessing module 253 may, for example, estimate a horizontal positionrelatively closer to sensor 552 by an amount proportional to therelative difference between respective amounts of energy. The sensorprocessing module 253 may perform a similar estimation utilizing sensors551 and 555. Various horizontal position estimations may then beaveraged. Alternatively for example, respective energies for the leftspeaker 531 sensors may be averaged, respective energies for the rightspeaker 533 sensors may be averaged, and such left and right speakeraverage energies may then be utilized to determine a horizontal pointinglocation. The sensor processing module 253 may then, for example,perform a similar pointing direction estimate in the vertical direction.

In another exemplary scenario, a calibration procedure may be performedto determine an expected sensor energy level (e.g., absolute orrelative) when the user is pointing at the sensor. In such a scenario,combined with a gain pattern and user (or pointing device) locationrelative to the television 501, a first line (e.g., a circle or arc) maybe drawn around a first sensor 552. Similarly, a second line (e.g., acircle or arc) may be drawn around a second sensor 553, and theintersection of the first and second lines utilized as an estimate ofpointing location. Additional lines associated with other sensors mayalso be utilized. Such additional lines may, for example, be utilizedwhen selecting between multiple line intersections and/or for greateraccuracy or resolution. Note that such line intersection solution may beapplied to any of the previously discussed scenarios (e.g., asillustrated in FIGS. 3-4). A non-limiting example of this was presentedin the discussion of FIG. 3, and another example will be provided in thefollowing discussion of FIG. 7.

After determining on-screen pointing location, the television controldevice 520 may communicate information of such determined location invarious manners. For example and without limitation, the sensorprocessing module 253 of the television control device 200 may utilizethe television interface module 235 to communicate information of suchon-screen pointing location to the television 501 for presentation tothe user. Also for example, the sensor processing module 253 may utilizethe user interface module 240 to communicate information of suchon-screen pointing location to the user (e.g., on a display of thetelevision control device 200). Such communication will also beaddressed in the discussions of FIGS. 9-10.

As discussed above, pointing sensors may be incorporated into thetelevision system off-television (i.e., placed separately in stand-alonehousings, integrated with other apparatus, attached to other apparatus,etc.). Another example of such off-television sensor placement ispresented in FIG. 6. In particular, the screen pointing sensors may beintegrated into the television receiver. FIG. 6 is a diagramillustrating an exemplary television system 600 with television receiversensors in accordance with various aspects of the present invention.

The television system 600 includes a television 601 comprising atelevision screen 603. The television system 600 also includes atelevision controller 620 (or other pointing device) pointing to anon-screen pointing location 630 along a pointing line 625 between thetelevision controller 620 and the on-screen pointing location 630. Thetelevision controller 620 may, for example, share any or all aspectswith the exemplary television controllers 160, 161, 200, 320, 420 and520 discussed previously and with all other television controllersdiscussed herein. Accordingly, various aspects of the television controldevice 620 will be explained herein with reference to various componentsof the exemplary television control device 200 illustrated in FIG. 2.The television control device 620 may, for example, be communicativelycoupled directly to the television 601 via a communication link (notillustrated). The television control device 620 may also, for example,be communicatively coupled directly to the television receiver 650 viacommunication link 653. The television control device 620 mayadditionally, for example, be communicatively coupled indirectly to thetelevision 601 via the television receiver 650 through communicationlinks 651 and 652. Accordingly, various aspects of the televisioncontrol device 620 will be explained herein with reference to variouscomponents of the exemplary television control device 200 illustrated inFIG. 2.

The television system 600 also comprises a television receiver 650 thatis communicatively coupled to the television 601 via a communicationlink 651 (e.g., a two-way communication link providing video informationto the television 601 and/or communicating sensor information and/orscreen pointing information with the television 601). The televisionreceiver 650 comprises an array of screen pointing sensors. A portion ofthe sensors are labeled (661-665) for discussion purposes. Note thatsuch sensors may be arranged in any of a variety of configurations(e.g., matrix configuration, border configuration, placed only at thefront corners, etc.). The pointing sensors may, for example, beintegrated into the television receiver 650 and/or attached to thetelevision receiver 650 in any of a variety of manners (e.g., in anymanner similar to those discussed previously with regard to thetelevisions and/or television system components discussed previously).

Note that in various exemplary scenarios, the television control device620 (e.g., a user interface module 240) may receive additional sensorinformation from other sensors via the television communication line 653and/or other communication links. The exemplary television controldevice 620 is also communicatively coupled to the television receiver650 via a communication link 652.

The exemplary television receiver 650 comprises an array of sensorsintegrated into the television receiver 650. For example, the televisionreceiver 650 comprises a lower left sensor 661, upper left sensor 662,upper right sensor 663, lower right sensor 664 and center sensor 665. Asdiscussed above, any of a variety of sensor types may be utilized,non-limiting examples of which include light sensors or photo detectors(e.g., photo diodes), RF sensors (e.g., antenna elements), acousticsensors (e.g., microphones), etc.

The exemplary television receiver 650 may be positioned around thetelevision 601 in any of a variety of manners. For example, thetelevision receiver 650 (and thus the sensors) may be positioned aroundthe television 601 in an orientation such that the front face of thetelevision receiver 650 (and thus the sensors) is generally in the sameplane as the television screen 603. Such placement is not necessary, butmay be advantageous from an accuracy perspective. In such an exemplaryscenario, on-screen pointing location may be determined in a mannersimilar to the interpolation and/or gain pattern intersection discussedabove with regard to off-screen and/or on-screen sensors. Note thatsince the locations of the sensors are likely to be inconsistent betweenvarious television system configurations (i.e., it is unlikely thatevery user will place/position the television receiver 650 in the samemanner), a calibration procedure may be implemented (e.g., by thecalibration module 251). Such calibration was discussed previously andwill also be revisited below.

In an exemplary configuration, one or more photo detectors (e.g., photodiodes) and/or antenna elements (e.g., individual antennas or elementsof an antenna array) may be incorporated into the faceplate of thetelevision receiver 650. Note that additional sensors positioned awayfrom the television receiver 650 may also be utilized (e.g., any of thepreviously discussed sensor placements).

For example, in a photo detector implementation, passive photo detectorsmay receive varying amounts of respective light energy depending on thepointing direction of a light source (e.g., a directional light sourceof the television control device 620) aimed at the screen. As discussedpreviously, directed energy (e.g., light, RF, acoustic, etc.) may betransmitted in a pattern (or envelope), so even if a pointing device ispointed to a location on the television screen 630 along pointing line625, sensors off-screen (e.g., sensors integrated into the televisionreceiver 650) may still receive energy from the transmission (albeitlikely not with the same intensity at which energy is delivered alongthe pointing line 625). Also for example, received signals (e.g., pulsedsignals) may arrive at different sensors at different respectivetimes/phases (e.g., being indicative of relative position and/orpointing direction, which may also be utilized in a pointingdetermination).

In a photo detector implementation (e.g., utilizing photo diodes), photodiodes may, for example, be tuned to react to particular lightfrequencies to reduce interference from output pixel light and/orassociated reflections, ambient light, room lighting, etc. As anon-limiting example, photo diodes may be tuned to detect light that isnot visible to the human eye, visible light frequencies that arerelatively rare, light patterns that are unlikely to occur in atelevision program (e.g., particular pulse codes), etc. In one example,the photo detectors integrated with the television receiver 650 maycomprise photo diodes that operate to detect energy from a laser pointeror directed infrared energy from the television control device 620 (orother pointing device). Note that analogously to the on-screen sensorsdiscussed previously, various aspects may comprise mounting (e.g.,adhering) sensors to various television receiver 650 locations and/or tovarious off-receiver components. Such sensor installation may, forexample, occur at the factory or after-market by a technician or user.

In an antenna element implementation, an array of antenna elements maybe positioned at locations on the television receiver 650 (e.g., only onthe television receiver 650 and/or at locations around the televisionreceiver 650). In an RF antenna implementation, passive antennas (orelements of an overall antenna matrix) may receive varying amounts ofrespective RF energy depending on the pointing direction of adirectional RF source pointed at a location on the screen. Also forexample, received signals (e.g., pulsed signals) may arrive at differentantennas at different respective times/phases (e.g., being indicative ofrelative position and/or pointing direction, which may also be utilizedin a pointing determination). Note that analogously to the on-screensensors discussed previously, various aspects may comprise mounting(e.g., adhering) sensors to the television receiver 650. Such sensorinstallation may, for example, occur at the factory or after-market by atechnician or user.

In an exemplary scenario, a user may point a pointing device (e.g., theremote control device 620, a laser pointer, directional RF transmitter,specifically designed eyewear, a mobile computing device, a mobilecommunication device, a gesture tracking device or glove, etc.) at thetelevision screen 603, where the pointing device directs transmittedenergy (e.g., light and/or RF energy and/or acoustic energy) at aparticular location on the television screen 603 to which the user ispointing with the pointing device. Note that such transmitted energywill likely be transmitted directionally and be associated with anintensity or gain (or energy) pattern with the highest intensity at thecenter of the pattern (i.e., along the pointing line 625) and decreasingas a function of angle from the center of the pattern. Such a gainpattern was discussed previously in the discussion of FIG. 4.

In such an exemplary scenario, each sensor of the sensors integratedinto the television receiver 650 off-television will likely receive somerespective amount of energy. For example, along a particular axis, thesensor nearest to the screen pointing location 630 (i.e., along thepointing line 625) will likely receive the highest amount of energy, asensor next nearest to the screen pointing location 630 will likelyreceive a next highest range of energy, and sensors away from thepointing location 630 will likely receive progressively less amounts ofenergy from the pointing device 620, as a function of distance from thepointing location 630 and/or angle off the pointing line 625 (e.g.,until such energy is lost in the noise floor). For example, sensor 662is nearest to the pointing location 630 and will likely receive thehighest energy, sensors 661 and 663 are further from the pointinglocation 630, etc., and so on.

Note that in the implementation illustrated in FIG. 6, in particularsince there are a relatively low number of sensors, signals from a samesensor may be utilized in determining multiple axes of pointinglocation. As mentioned previously, a calibration procedure may beperformed when the system 600 is configured to assist in such pointingdetermination.

In an exemplary scenario, the television control device 620 (e.g., theuser interface module 240 of the television control device 200illustrated in FIG. 2) may receive signals indicative of the energyreceived by the sensors of the television receiver 650 (e.g., via thecommunication link 652 between the television control device 620 and thetelevision receiver 650 and/or via a communication link directly betweenthe television control device 620 and the sensors). The televisionreceiver 650 may receive such signals in various manners, depending onthe degree of integration of such sensors into the television receiver650 and/or various components of the television system 600. For example,in an exemplary scenario where the sensors are fully integrated into thetelevision receiver 650, the television control device 620 may receivesuch signals via communication link 652. Also for example, in a scenariowhere various sensors are off the television receiver 650, thetelevision control device 620 may receive information from such sensorsvia direct communication link or via communication link with the variouscomponents with which such sensors are integrated.

The communication module 230 may then, for example, provide informationof such received sensor signals to the sensor processing module 253 forprocessing. The sensor processing module 253 may then, for example,operate to process such information to determine the screen pointinglocation. The sensor processing module 253 may perform such processingin any of a variety of manners, non-limiting examples of which will beprovided below.

In an exemplary scenario, the sensor processing module 253 may operateto estimate a position between sensor positions based on relative sensorenergy. For example, in the horizontal dimension, sensor 662 maycorrespond to a relatively high amount of energy, and sensor 663 maycorrespond to a relatively low amount of received energy. The sensorprocessing module 253 may, for example, estimate a horizontal positionrelatively closer to sensor 662 by an amount proportional to therelative difference between respective amounts of energy. The sensorprocessing module 253 may perform a similar estimation utilizing sensors661 and 664. Various horizontal position estimations may then beaveraged. Alternatively for example, respective energies for the leftside sensors 661, 662 may be averaged, respective energies for the rightside sensors 663, 664 sensors may be averaged, and such left and rightspeaker average energies may then be utilized (e.g., in conjunction withenergy pattern characteristics) to determine a horizontal pointinglocation. The sensor processing module 253 may then, for example,perform a similar pointing direction estimate in the vertical direction.Such horizontal and/or vertical positions may, for example, betranslated between respective locations/directions of the sensorarrangement and respective locations/directions of the television screen603. Calibrations procedures may, for example, be utilized to establishthe spatial relationship between the sensor positioning and on-screenlocation.

In another exemplary scenario, a calibration procedure may be performedto determine an expected sensor energy level (e.g., absolute orrelative) when the user is pointing at the sensor (and/or other knownlocations). In such a scenario, combined with a gain pattern and user(or pointing device) location relative to the television 601, a firstline (e.g., a circle or arc) may be drawn around a first sensor 662.Similarly, a second line (e.g., a circle or arc) may be drawn around asecond sensor 663, and the intersection of the first and second linesutilized as an estimate of pointing location. Additional linesassociated with other sensors may also be utilized. Such additionallines may, for example, be utilized when selecting between multiple lineintersections or to increase accuracy and/or resolution of the pointingdetermination. Note that such line intersection solution may be appliedto any of the previously discussed scenarios (e.g., as illustrated inFIGS. 3-5) or other scenarios discussed herein. A non-limiting exampleof this was presented in the discussion of FIG. 3, and another examplewill be provided in the following discussion of FIG. 7.

After determining on-screen pointing location, the television receiver650 may communicate information of such determined location in variousmanners. For example and without limitation, the sensor processingmodule 253 of the television control device 200 may utilize thetelevision interface module 235 to communicate information of suchon-screen pointing location to the television 601 for presentation tothe user on the television screen 603. Also for example, the sensorprocessing module 253 may utilize the user interface module 240 tocommunicate information of such on-screen pointing location to the user(e.g., on a display of the television control device 620). Suchcommunication will also be addressed in the discussions of FIGS. 9-10.

Various aspects of the present invention may also, for example, includeone or more sensors incorporated into the pointing device (e.g., thetelevision controller 200). FIG. 7 is a diagram illustrating anexemplary television system 700 utilizing pointing device sensors inaccordance with various aspects of the present invention.

The exemplary television system 700 includes a television 701 having atelevision screen 703. The television system 700 also includes atelevision controller 720 (or other pointing device) pointing to anon-screen pointing location 730 along a pointing line 725 between thetelevision controller 720 and the on-screen pointing location 730. Thetelevision controller 720 may, for example, share any or all aspectswith the exemplary television controllers 160, 161, 200, 320, 420, 520and 620 discussed previously and with all other television controllersdiscussed herein. Accordingly, various aspects of the television controldevice 720 will be explained herein with reference to various componentsof the exemplary television control device 200 illustrated in FIG. 2.

The television control device 720 may, for example, be communicativelycoupled directly to the television 701 via a communication link 753. Thetelevision control device 720 may also, for example, be communicativelycoupled directly to the television receiver 750 via communication link752. The television control device 720 may additionally, for example, becommunicatively coupled indirectly to the television 701 via thetelevision receiver 750 through communication links 751 and 752.Accordingly, various aspects of the television control device 720 willbe explained herein with reference to various components of theexemplary television control device 200 illustrated in FIG. 2.

The television system 700 also comprises a television receiver 750 thatis communicatively coupled to the television 701 via a communicationlink 751 (e.g., a two-way communication link providing video informationto the television 701 and/or receiving sensor information from thetelevision 701). The exemplary television receiver 750 is alsocommunicatively coupled to the television controller 720 via acommunication link 752.

In such a configuration, sensor information may be communicated to thetelevision control device 720 (e.g., via internal communication link).Such information may then be communicated to the sensor processingmodule 253 for the determination of an on-screen pointing location.

In the exemplary configuration, the television 701 includes eightemitters (e.g., light emitters, RF transmitters, etc.) located aroundthe border of the television screen 703. Note that such emitters may bepositioned anywhere proximate the television system 700. For example,the television 701 includes a first emitter 711, second emitter 712,third emitter 713, fourth emitter 714, fifth emitter 715, sixth emitter716, seventh emitter 717 and eighth emitter 718. Such emitters may eachemit a signal that may be received at sensors on-board the televisioncontrol device 720. Such sensors may, for example, make up a directionalreceiver. In such a configuration, the controller 720 (or other pointingdevice) may be pointed to a location 730 on the screen 703 along apointing line 725. With such an orientation and a directional signalreception pattern, the sensors on-board the controller 720 will receivethe emitted signals at respective signal levels. Such sensor signals maythen be processed in a manner similar to the manners discussed above todetermine the on-screen pointing direction for the pointing device 720.

For example, through a calibration procedure, it may be known that thepointing device at a particular location should receive a particularamount of energy from each of the emitters 711-718 when pointed directlyat such emitters (or at some other known location). In such a scenario,the pointing device (e.g., the user interface module 240 of thetelevision control device 200) may measure respective signal energiesreceived from each of the emitters (e.g., each distinguishable byfrequency, coding, timing and/or timeslotting, etc.) and communicatesuch information to the television receiver 750.

The sensor processing module 253 may, for example, select a firstemitter 712 (e.g., the emitter corresponding to the highest energyreceived at the pointing device). The sensor processing module 253 maythen process the location of the pointing device, the receive gainpattern for the pointing device, and the energy received from the firstemitter 712 to determine a first figure (e.g., an arc 752) along whichthe pointing device, if pointed, would be expected to receive themeasured energy. Similarly, the sensor processing module 253 may performsuch a procedure for a second emitter 711 resulting in a second figure(e.g., an arc 751). The intersection of such arcs may be utilized as anestimate of on-screen pointing location. Additionally, for accuracy orfor selecting between multiple intersection points, should they occur,the sensor processing module 253 may perform such a procedure for athird emitter 714 resulting in a third figure (e.g., an arc 754), and soon. The intersection of the three arcs 752, 751, 754 may then beutilized as an estimate of on-screen pointing location.

Alternatively, the solution need not be based on a known position(location) of the pointing device, nor on absolute received energylevels. In such a scenario, differences in received energy from thevarious emitters may be processed with or without position informationof the on-screen pointing device. For example, the pointing device 720may have six degrees of freedom (e.g., three positional degrees offreedom and three orientational degrees of freedom). In such a scenario,if the position and orientation of the television 701 are known, theunknown six degrees of freedom for the pointing device 720 may beascertained by processing six known values related to such six degreesof freedom (e.g., related by a known signal energy pattern). In such ascenario, measurements associated with six emitters on the television(and potentially more) may be utilized to solve for the six degrees offreedom of the pointing device 720.

The above-mentioned exemplary scenarios were presented to illustratenumerous manners in which the television control device 720 (e.g.,sensor processing module 253) may operate to determine on-screenpointing location. Such examples are merely exemplary and thus the scopeof various aspects of the present invention should not be limited by anyparticular characteristics of such examples unless explicitly claimed.

As discussed above, the calibration module 251 of the television controldevice 200 may operate to perform calibration operations. Suchcalibrating may be performed in any of a variety of manners. For exampleand without limitation, calibration may be utilized to determineexpected received energy when transmitters and receivers are located andoriented in a particular manner. For example, a non-limiting example ofa calibration procedure may comprise presenting an on-screen target atvarious locations and measuring respective sensor signals received whenthe pointing device is being pointed at such targets. Also for example,a calibration procedure may comprise directing a user (e.g., using theuser interface module 240) to point to each of a plurality of sensors todetermine an expected amount of received energy when the user ispointing directly at such sensors.

As mentioned previously, signal energy (or gain) pattern may be utilizedin various on-screen pointing determinations. Such an energy (or gain)pattern may be predefined for a particular pointing device (e.g., at thefactory), but may also be measured by the television control device 200.In a non-limiting example, the calibration module 251 may direct theuser to utilize a pointing device to point to a location on the screenand process information received from multiple sensors (e.g., embeddedin the screen, embedded in the television around the border of thescreen, located in off-television devices, located on the televisioncontrol device 720, located in the pointing device, etc.) to develop acustom gain pattern for the particular pointing device (e.g., for thetelevision control device 200). For example, such calibration maydetermine the shape of the gain pattern, the signal energy falloffcharacteristics, etc.

Various aspects discussed above included the processing of positioninformation. In such exemplary cases, the television control device 200may comprise one or more location modules 252 that operate to determinerelevant position information. The location module 252 may operate toperform such location determining (e.g., of the user or pointing deviceand/or the television) in any of a variety of manners. For example, thelocation module 252 may utilize a communication interface module 210,220 to receive position information (e.g., of the television controldevice 200 or other pointing device) from an external source of suchinformation (e.g., global positioning system, cellular triangulationsystem, home triangulation system, etc.).

Also for example, the location module 252 may receive positioninformation from internal components of the television control device200 (e.g., where such television control device 200 hasposition-determining capability). For example, in a non-limitingexemplary scenario, where the television control device 200 is ahandheld computer, such computer may comprise GPS (or A-GPS) capabilityto determine its position. In such a scenario, the television controldevice 200 location module 252 may wirelessly communicate information ofthe television control device's position to the sensor processing module253.

Additionally for example, the location module 232 may operate to processsensor information to determine location of the pointing device (e.g.,location in relation to the television screen). For example, asmentioned previously, a signal (e.g., a pulse) transmitted from apointing device to the television (or vice versa) will arrive atdifferent sensors at different points in time depending on therespective distance from the pointing device to each sensor. Thelocation module 232 may process such time-of-arrival information atvarious sensors to determine the position of the pointing devicerelative to the television. Similarly, in a scenario including signalemitters associated with the television and sensors on the pointingdevice, simultaneously transmitted signals (or signals transmitted witha known temporal pattern) from different emitters will arrive at thepointing device at different respective times depending on the positionof the pointing device relative to such emitters. Alternatively, thelocation module 232 may also operate to process phase differenceinformation (in addition to timing information or instead of suchinformation) to determine pointing device location.

Once the television control device 200 (e.g., the sensor processingmodule 253) determines an on-screen pointing location, the televisioncontrol device 200 may utilize such information in any of a variety ofmanners. For example and without limitation, the sensor processingmodule 253 may operate to generate information of the determinedon-screen pointing location, and one or more modules of the televisioncontrol device 200 may operate to communicate a signal (e.g., to atelevision, television receiver, other display device, U/I modules 240of the television control device 200, etc.) that comprisescharacteristics that cause presentation of a visual indication (e.g., onthe television screen, controller screen, other display, etc.) toindicate to the user the on-screen location to which the televisioncontrol device 200 has determined the user is pointing. Such a visualindication may, for example, comprise characteristics of a cursor orother graphical construct, bright spot, highlighting, color variation,brightness variation, etc. For example, the television 701 or televisioncontrol device 720 may operate to overlay such indication on videocontent (e.g., television programming) being presented to the user(e.g., presented on the television screen, presented on a screen of thetelevision controller, etc.).

Additionally for example, the sensor processing module 253 may provideinformation of the determined on-screen pointing location to one or moreother modules of the television control device 200 (e.g., the processingmodule 250 and/or other modules thereof) to identify an object in videocontent (e.g., television programming) to which a user is pointing. Insuch an exemplary scenario, one or more modules of the televisioncontrol device 200 may operate to communicate signals (e.g., to atelevision, other modules of the television controller having a screen,other display device, etc.) that cause highlighting of an object towhich the user is pointing and/or provide information regarding suchobject.

Further for example, various modules of the television control device200 (e.g., the processor module 250) may operate to communicateon-screen pointing location information to television system componentsseparate from the television (e.g., to a television receiver, videorecorder, remote programming source, communication networkinfrastructure, advertising company, provider of goods and/or services,etc.).

FIG. 2 provided a diagram illustrating an exemplary television controldevice 200 in accordance with various aspects of the present invention.FIG. 8 provides another diagram illustrating an exemplary televisioncontrol device 800 in accordance with various aspects of the presentinvention. The exemplary television control device 800 may share any orall aspects with any of the television control devices discussed hereinand illustrated in FIGS. 1-7. For example, the exemplary televisioncontrol device 800 (or various modules thereof) may operate to performany or all functionality discussed herein. As with the exemplarytelevision control device 200, the components of the exemplarytelevision control device 800 may be co-located a single housing.

For example, the television control device 800 comprises a processor830. Such a processor 830 may, for example, share any or allcharacteristics with the processor 250 discussed with regard to FIG. 2.Also for example, the television control device 800 comprises a memory840. Such memory 840 may, for example, share any or all characteristicswith the memory 260 discussed with regard to FIG. 2.

Also for example, the television control device 800 may comprise any ofa variety of user interface module(s) 850. Such user interface module(s)850 may, for example, share any or all characteristics with the userinterface module(s) 240 discussed previously with regard to FIG. 2. Forexample and without limitation, the user interface module(s) 850 maycomprise: a display device, a camera (for still or moving pictureacquisition), a speaker, an earphone (e.g., wired or wireless), amicrophone, a video screen (e.g., a touch screen display), a vibratingmechanism, a keypad, a remote control interface, and/or any of a varietyof other user interface devices (e.g., a mouse, a trackball, a touchpad, touch screen, light pen, game controlling device, etc.).

The exemplary television control device 800 may also, for example,comprise any of a variety of communication modules (805, 806, and 810).Such communication module(s) may, for example, share any or allcharacteristics with the communication interface module(s) 210, 220discussed previously with regard to FIG. 2. For example and withoutlimitation, the communication interface module(s) 810 may comprise: aBluetooth interface module; an IEEE 802.11, 802.15, 802.16 and/or 802.20module; any of a variety of cellular telecommunication interface modules(e.g., GSM/GPRS/EDGE, CDMA/CDMA2000/1x-EV-DO, WCDMA/HSDPA/HSUPA,TDMA/PDC, WiMAX, etc.); any of a variety of position-relatedcommunication interface modules (e.g., GPS, A-GPS, etc.); any of avariety of wired/tethered communication interface modules (e.g., USB,Fire Wire, RS-232, HDMI, component and/or composite video, Ethernet,wireline and/or cable modem, etc.); any of a variety of communicationinterface modules related to communicating with external memory devices;etc. The exemplary television control device 800 is also illustrated ascomprising various wired 806 and/or wireless 805 front-end modules thatmay, for example, be included in the communication interface modulesand/or utilized thereby.

The exemplary television control device 800 may also comprise any of avariety of signal processing module(s) 890. Such signal processingmodule(s) 890 may, for example, be utilized to assist in processingvarious types of information discussed previously (e.g., with regard tosensor processing, position determination, video processing, imageprocessing, audio processing, general user interface information dataprocessing, etc.). For example and without limitation, the signalprocessing module(s) 890 may comprise: video/graphics processing modules(e.g. MPEG-2, MPEG-4, H.263, H.264, JPEG, TIFF, 3-D, 2-D, MDDI, etc.);audio processing modules (e.g., MP3, AAC, MIDI, QCELP, AMR, CMX, etc.);and/or tactile processing modules (e.g., Keypad I/O, touch screenprocessing, motor control, etc.).

Various aspects of the present invention were previously exemplified bynon-limiting illustrations and described in terms of operationsperformed by various modules of the television. Various aspects of thepresent invention will now be illustrated in the form of method flowdiagrams.

FIG. 9 is a flow diagram 900 illustrating the generation of on-screenpointing information (e.g., in a television control device) inaccordance with various aspects of the present invention. The exemplarymethod 900 may, for example, share any or all characteristics with thetelevision control device operation discussed previously. For example,the exemplary method 900 may be implemented by any or all of thetelevision control devices (e.g., 160, 161, 200, 220, 320, 420, 520,620, 720 and 800) discussed previously. Conversely, the exemplary method900 may comprise any or all functional aspects discussed previously withregard to such exemplary television control devices.

The exemplary method 900 may begin executing at step 905. The exemplarymethod 900 may begin executing in response to any of a variety of causesand/or conditions. For example and without limitation, the method 900may begin executing in response to a user command to begin, detecteduser interaction with a pointing device (e.g., a television controller),detected user presence in the vicinity, detected user interaction with atelevision implementing the method 900, etc. Also for example, themethod 900 may begin executing in response to a television presentingprogramming or other video content for which on-screen pointing isenabled and/or relevant.

The exemplary method 900 may, for example at step 910, comprisereceiving pointing sensor information. For example and withoutlimitation, step 910 may comprise any or all sensor informationreceiving characteristics described previously with regard the variousmodules of the exemplary television control devices illustrated in FIGS.1-8 and discussed previously. For example, step 910 may share any or allsensor information receiving characteristics discussed previously withregard to at least the user interface module 240, television interfacemodule 235, processor module 250, communication interface modules 210,220, sensor processing module 253, location module 252 and calibrationmodule 251.

Step 910 may, for example, comprise receiving sensor information from(or associated with) sensors integrated in the television controldevice. Also for example, step 910 may comprise receiving sensorinformation from (or associated with) off-controller sensors (e.g.,integrated with or attached to a television, off-television sensors,sensors integrated with a pointing device different from the televisioncontrol device, sensors integrated with a television receiver, etc. Asdiscussed previously, such sensors may comprise any of a variety ofcharacteristics, including without limitation, characteristics of lightsensors, RF sensors, acoustic sensors, active and/or passive sensors,etc.

In general, step 910 may comprise receiving pointing sensor information.Accordingly, the scope of various aspects of the present inventionshould not be limited by characteristics of any particular manner ofreceiving pointing sensor information unless explicitly claimed.

The exemplary method 900 may, at step 920, comprise processing receivedsensor information (e.g., as received at step 910) to determine alocation on a screen of the television to which a user is pointing(e.g., pointing with a pointing device). For example and withoutlimitation, step 920 may comprise any or all pointing locationprocessing characteristics described previously with regard the variousmodules of the exemplary television controllers illustrated in FIGS. 1-8and discussed previously. For example, step 920 may share any or allpointing location determining characteristics discussed previously withregard to at least the processor module 250, sensor processing module253, location module 252 and calibration module 251.

Step 920 may, for example, comprise determining on-screen pointinglocation in any of a variety of manners. For example, step 920 maycomprise determining on-screen pointing location based on a location ofa selected sensor, based on interpolation between sensor locations(e.g., linear and/or non-linear interpolation), based on determiningenergy pattern intersection(s), etc. Many examples of such determiningwere provided previously.

In general, step 920 may comprise processing received sensor information(e.g., independently and/or in conjunction with other information) todetermine a location on a screen of the television to which a user ispointing (e.g., while the television is presenting programming to theuser). Accordingly, the scope of various aspects of the presentinvention should not be limited by characteristics of any particularmanner of performing such processing unless explicitly claimed.

The exemplary method 900 may, at step 930, comprise generatinginformation indicative of a determined on-screen pointing location(e.g., as determined at step 920). For example and without limitation,step 930 may comprise any or all pointing location informationgeneration characteristics described previously with regard the variousmodules of the exemplary television control devices illustrated in FIGS.1-8 and discussed previously. For example, step 930 may share any or allinformation generation characteristics discussed previously with regardto at least the processor module 250, sensor processing module 253,location module 252, calibration module 251, television interface module235, user interface module 240 and/or communication interface modules210, 220.

Step 930 may, for example, comprise generating such information in anyof a variety of manners. For example, step 930 may comprise generatingon-screen pointing location data to communicate to internal modules ofthe television control device, to equipment external to the televisioncontrol device (e.g., to the television and/or television receiver), totelevision network components, to a television programming source, etc.Such information may, for example, be communicated to various systemcomponents and may also be presented to the user (e.g., utilizing visualfeedback displayed on a screen of a television, television controller,etc.). Such information may, for example, be generated in the form ofscreen coordinates, identification of a video content object (e.g., aprogramming object or person) to which an on-screen pointing locationcorresponds, generation of an on-screen cursor or highlight or othergraphical feature, etc.

In general, step 930 may comprise generating information indicative of adetermined on-screen pointing location. Accordingly, the scope ofvarious aspects of the present invention should not be limited bycharacteristics of any particular manner of generating such informationunless explicitly claimed.

The exemplary method 900 may, at step 995, comprise performing continuedprocessing. Such continued processing may comprise characteristics ofany of a variety of types of continued processing, various examples ofwhich were presented previously. For example and without limitation,step 995 may comprise looping execution flow back up to any earlier step(e.g., step 910). Also, in a non-limiting exemplary scenario, step 995may comprise presenting a graphical feature on a television controldevice screen indicative of where the user is pointing on a televisionscreen. In another exemplary scenario, step 995 may comprisecommunicating information to a television that causes the television tooutput a graphical feature on the television screen indicative of wherethe user is pointing (e.g., such information may comprisecharacteristics that cause the television to overlay such graphicalindication on programming being presented on the television screen.Additionally for example, step 995 may comprise presenting (or causingthe presentation of) visual feedback indicia of the on-screen pointinglocation for a user. Further for example, step 995 may comprisecommunicating information of the on-screen pointing location to systemcomponents external to the television control device implementing themethod 900 (e.g., to a television, television receiver, anothertelevision controller, etc.). Further for example, step 995 may compriseutilizing the on-screen pointing information to identify a video contentobject (e.g., an object presented in television programming) to which auser is pointing, etc.

In general, step 995 may comprise performing continued processing.Accordingly, the scope of various aspects of the present inventionshould not be limited by characteristics of any particular manner ofperforming continued processing unless explicitly claimed.

Turning next to FIG. 10, such figure is a flow diagram 1000 illustratingthe generation of on-screen pointing information (e.g., in a televisioncontrol device) in accordance with various aspects of the presentinvention. The exemplary method 1000 may, for example, share any or allcharacteristics with the television control device operation discussedpreviously (e.g., in reference to FIGS. 1-9).

The exemplary method 1000 may begin executing at step 1005. Step 1005may, for example, share any or all characteristics with step 905 of theexemplary method 900 illustrated in FIG. 9 and discussed previously.

The exemplary method 1000 may, for example at step 1008, compriseperforming a calibration procedure with the user. Such a calibrationprocedure may, for example, be performed to develop a manner ofprocessing received sensor information to determine on-screen pointinglocation. Step 1008 may, for example, comprise any or all calibrationaspects discussed previously (e.g., with reference to the calibrationmodule 251).

The exemplary method 1000 may, for example at step 1010, comprisereceiving pointing sensor information. For example and withoutlimitation, step 1010 may comprise any or all sensor informationreceiving characteristics described previously with regard the variousmodules of the exemplary television control devices illustrated in FIGS.1-8 and FIG. 9 (e.g., step 910) and discussed previously.

The exemplary method 1000 may, for example at step 1015, comprisedetermining user position (e.g., determining position of a user pointingdevice). For example and without limitation, step 1015 may comprise anyor all position determining characteristics discussed previously withregard to FIGS. 1-9. Note that position may also, for example, includeorientation.

For example, step 1015 may share any or all position determiningcharacteristics discussed previously with regard to at least theprocessor module 250, sensor processing module 253, location module 252and calibration module 251. For example, step 1015 may comprisedetermining user position based, at least in part, on received sensorsignals. Also for example, step 1015 may comprise determining userposition based, at least in part, on position information received fromone or more systems external to the television control deviceimplementing the method 1000.

In general, step 1015 may comprise determining user position (e.g.,pointing device position). Accordingly, the scope of various aspects ofthe present invention should not be limited by characteristics of anyparticular manner of determining user position unless explicitlyclaimed.

The exemplary method 1000 may, for example, at step 1020, compriseprocessing received sensor information (e.g., as received at step 1010)and/or user position information (e.g., as determined at step 1015) todetermine a location on a screen of the television to which a user ispointing (e.g., pointing with the television control device implementingthe method or other pointing device). For example and withoutlimitation, step 1020 may comprise any or all pointing locationdetermination characteristics described previously with regard thevarious modules of the exemplary television control devices illustratedin FIGS. 1-8 and FIG. 9 (e.g., step 920) and discussed previously. Forexample, step 1020 may share any or all pointing location determiningcharacteristics discussed previously with regard to at least theprocessor module 250, sensor processing module 253, location module 252and calibration module 251.

Step 1020 may, for example, comprise determining on-screen pointinglocation in any of a variety of manners. For example, step 1020 maycomprise determining on-screen pointing location based on a location ofa selected sensor, based on location of the pointing device, based oninterpolation between sensor locations (e.g., linear and/or non-linearinterpolation), based on energy pattern intersection points, etc. Manyexamples of such determining were provided previously.

In general, step 1020 may comprise processing received sensorinformation and/or user position information to determine a location ona screen of the television to which a user is pointing (e.g., pointingwith the television control device implementing the method 1000 or otherpointing device). Accordingly, the scope of various aspects of thepresent invention should not be limited by characteristics of anyparticular manner of performing such processing unless explicitlyclaimed.

The exemplary method 1000 may, at step 1030, comprise generatinginformation indicative of a determined on-screen pointing location(e.g., as determined at step 1020). For example and without limitation,step 1030 may comprise any or all information generation characteristicsdescribed previously with regard the various modules of the exemplarytelevision control devices illustrated in FIGS. 1-8 and FIG. 9 (e.g.,step 930) and discussed previously. For example, step 1030 may share anyor all information generation characteristics discussed previously withregard to at least the processor module 250, sensor processing module253, location module 252, calibration module 251, television interfacemodule 235, user interface module 240 and/or communication interfacemodules 210, 220.

The exemplary method 1000 may, at step 1095, comprise performingcontinued processing. Such continued processing may comprisecharacteristics of any of a variety of types of continued processing,various examples of which were presented previously. For example andwithout limitation, step 1095 may comprise looping execution flow backup to any earlier step (e.g., step 1008). Also, in a non-limitingexemplary scenario, step 1095 may comprise presenting a graphicalfeature on a television control device screen indicative of where theuser is pointing on a television screen. In another exemplary scenario,step 1095 may comprise communicating information to a television thatcauses the television to output a graphical feature on the televisionscreen indicative of where the user is pointing (e.g., such informationmay comprise characteristics that cause the television to overlay suchgraphical indication on programming being presented on the televisionscreen. Additionally for example, step 1095 may comprise presenting(and/or causing the presentation of) visual feedback indicia of theon-screen pointing location for a user. Further for example, step 1095may comprise communicating information of the on-screen pointinglocation to system components external to the television receiverimplementing the method 1000. Further for example, step 1095 maycomprise utilizing the on-screen pointing information to identify avideo content object (e.g., an object presented in televisionprogramming) to which a user is pointing, etc.

In general, step 1095 may comprise performing continued processing.Accordingly, the scope of various aspects of the present inventionshould not be limited by characteristics of any particular manner ofperforming continued processing unless explicitly claimed.

In summary, various aspects of the present invention provide a systemand method in a television controller (e.g., a television controldevice) for generating screen pointing information. While the inventionhas been described with reference to certain aspects and embodiments, itwill be understood by those skilled in the art that various changes maybe made and equivalents may be substituted without departing from thescope of the invention. In addition, many modifications may be made toadapt a particular situation or material to the teachings of theinvention without departing from its scope. Therefore, it is intendedthat the invention not be limited to the particular embodimentdisclosed, but that the invention will include all embodiments fallingwithin the scope of the appended claims.

What is claimed is:
 1. A method for identifying a user-selected objectin television programming, the method comprising: in a television havinga screen: presenting television programming, the television programmingcomprising user-selectable objects; determining, by sensors integratedinto the screen, an on-screen pointing location pointed to by a user ofthe television; and identifying a user-selectable object in thepresented television programming at which the user is pointing based, atleast in part, on the determined on-screen pointing location.
 2. Themethod of claim 1, comprising, in the television, receiving informationidentifying selectable objects in the presented television programming.3. The method of claim 2, wherein receiving information identifyingselectable objects in the presented television programming comprisesreceiving said information embedded in a same data stream as a streamcommunicating the presented television programming.
 4. The method ofclaim 2, wherein receiving information identifying selectable objects inthe presented television programming comprises receiving saidinformation in a data stream communicated in parallel with a streamcommunicating the presented television programming.
 5. The method ofclaim 2, wherein the information identifying selectable objects in thepresented television programming is communicated with the presentedtelevision programming without being requested by the television.
 6. Themethod of claim 2, wherein the information identifying selectableobjects in the presented television programming comprises timinginformation associated with the selectable objects.
 7. The method ofclaim 2, wherein the information identifying selectable objects in thepresented television programming comprises information definingrespective regions of the presented television programming associatedwith respective selectable objects in the presented televisionprogramming.
 8. The method of claim 7, wherein the information definingrespective regions of the presented television programming associatedwith the respective selectable objects comprises information describingmovement of the respective regions as a function of time.
 9. The methodof claim 1, wherein: the presented television programming isuser-stored; and information identifying selectable objects isuser-stored.
 10. The method of claim 1, comprising generating a useroutput indicating the identified user-selectable object.
 11. The methodof claim 1, comprising communicating information indicating theidentified user-selectable object to a television remote control device.12. A video display device comprising at least one module operable to,at least: present a video stream, the video stream comprisinguser-selectable objects; determine an on-screen pointing locationpointed to by a pointing device; and identify a user-selectable objectin the presented video stream at which the pointing device is pointingbased, at least in part, on the determined on-screen pointing location.13. The video display device of claim 12, wherein the at least onemodule is operable to determine the on-screen pointing location based ona time of arrival of an emission from the pointing device.
 14. The videodisplay device of claim 12, wherein the at least one module is operableto determine the on-screen pointing location based on a phase of anemission from pointing device.
 15. The video display device of claim 12,wherein the at least one module is operable to determine the on-screenpointing location based a gain pattern of an emission from the pointingdevice.
 16. The video display device of claim 12, wherein the pointingdevice is a remote control.
 17. The video display device of claim 12,wherein the pointing device is an eyewear device.
 18. A televisionreceiver for communicating with a television comprising: at least onemodule, located in the television receiver, operable to, at least:provide television programming to the television, the televisionprogramming comprising user-selectable objects; determine an on-screenpointing location on the television pointed to by a remote control; andidentify a user-selectable object in the presented televisionprogramming at which the remote control is pointing based, at least inpart, on the determined on-screen pointing location.
 19. The televisionreceiver of claim 18, wherein the at least one module is operable tooverlay a graphical feature coinciding with the identifieduser-selectable object in the presented television programming.
 20. Thetelevision receiver of claim 18, wherein the at least one module isoperable to communicate information indicating the identifieduser-selectable object to a television remote control device.